.. _`Logistic Regression`: .. _`org.sysess.sympathy.machinelearning.logisticregression`: Logistic Regression ``````````````````` .. image:: logistic_regression.svg :width: 48 Logistic regression of a categorical dependent variable Documentation ::::::::::::: Attributes ========== **coef_** Coefficient of the features in the decision function. `coef_` is of shape (1, n_features) when the given problem is binary. In particular, when `multi_class='multinomial'`, `coef_` corresponds to outcome 1 (True) and `-coef_` corresponds to outcome 0 (False). **intercept_** Intercept (a.k.a. bias) added to the decision function. If `fit_intercept` is set to False, the intercept is set to zero. `intercept_` is of shape (1,) when the given problem is binary. In particular, when `multi_class='multinomial'`, `intercept_` corresponds to outcome 1 (True) and `-intercept_` corresponds to outcome 0 (False). **n_iter_** Actual number of iterations for all classes. If binary or multinomial, it returns only 1 element. For liblinear solver, only the maximum number of iteration across all classes is given. .. versionchanged:: 0.20 In SciPy <= 1.0.0 the number of lbfgs iterations may exceed ``max_iter``. ``n_iter_`` will now report at most ``max_iter``. Definition :::::::::: Output ports ============ **model** model Model Configuration ============= **C** (C) Inverse of regularization strength; must be a positive float. Like in support vector machines, smaller values specify stronger regularization. **Class weights** (class_weight) Weights associated with classes in the form ``{class_label: weight}``. If not given, all classes are supposed to have weight one. The "balanced" mode uses the values of y to automatically adjust weights inversely proportional to class frequencies in the input data as ``n_samples / (n_classes * np.bincount(y))``. Note that these weights will be multiplied with sample_weight (passed through the fit method) if sample_weight is specified. .. versionadded:: 0.17 *class_weight='balanced'* **Dual Formulation** (dual) Dual (constrained) or primal (regularized, see also this equation ) formulation. Dual formulation is only implemented for l2 penalty with liblinear solver. Prefer dual=False when n_samples > n_features. **Fit intercept** (fit_intercept) Specifies if a constant (a.k.a. bias or intercept) should be added to the decision function. **Intercept scaling** (intercept_scaling) Useful only when the solver 'liblinear' is used and self.fit_intercept is set to True. In this case, x becomes [x, self.intercept_scaling], i.e. a "synthetic" feature with constant value equal to intercept_scaling is appended to the instance vector. The intercept becomes ``intercept_scaling * synthetic_feature_weight``. Note! the synthetic feature weight is subject to l1/l2 regularization as all other features. To lessen the effect of regularization on synthetic feature weight (and therefore on the intercept) intercept_scaling has to be increased. **Maximum iterations** (max_iter) Maximum number of iterations taken for the solvers to converge. **Multiclass** (multi_class) If the option chosen is 'ovr', then a binary problem is fit for each label. For 'multinomial' the loss minimised is the multinomial loss fit across the entire probability distribution, *even when the data is binary*. 'multinomial' is unavailable when solver='liblinear'. 'auto' selects 'ovr' if the data is binary, or if solver='liblinear', and otherwise selects 'multinomial'. .. versionadded:: 0.18 Stochastic Average Gradient descent solver for 'multinomial' case. .. versionchanged:: 0.22 Default changed from 'ovr' to 'auto' in 0.22. **Number of jobs** (n_jobs) Number of CPU cores used when parallelizing over classes if multi_class="ovr". Ignored when the solver is set to "liblinear" regardless of multi_class. If given -1 then all cores are used **Penalty** (penalty) Specify the norm of the penalty: - `None`: no penalty is added; - `'l2'`: add a L2 penalty term and it is the default choice; - `'l1'`: add a L1 penalty term; - `'elasticnet'`: both L1 and L2 penalty terms are added. .. warning:: Some penalties may not work with some solvers. See the parameter `solver` below, to know the compatibility between the penalty and solver. .. versionadded:: 0.19 l1 penalty with SAGA solver (allowing 'multinomial' + L1) **Random seed** (random_state) Used when ``solver`` == 'sag', 'saga' or 'liblinear' to shuffle the data. See random_state for details. **Solver** (solver) Algorithm to use in the optimization problem. Default is 'lbfgs'. To choose a solver, you might want to consider the following aspects: - For small datasets, 'liblinear' is a good choice, whereas 'sag' and 'saga' are faster for large ones; - For multiclass problems, only 'newton-cg', 'sag', 'saga' and 'lbfgs' handle multinomial loss; - 'liblinear' is limited to one-versus-rest schemes. - 'newton-cholesky' is a good choice for `n_samples` >> `n_features`, especially with one-hot encoded categorical features with rare categories. Note that it is limited to binary classification and the one-versus-rest reduction for multiclass classification. Be aware that the memory usage of this solver has a quadratic dependency on `n_features` because it explicitly computes the Hessian matrix. .. warning:: The choice of the algorithm depends on the penalty chosen. Supported penalties by solver: - 'lbfgs' - ['l2', None] - 'liblinear' - ['l1', 'l2'] - 'newton-cg' - ['l2', None] - 'newton-cholesky' - ['l2', None] - 'sag' - ['l2', None] - 'saga' - ['elasticnet', 'l1', 'l2', None] .. note:: 'sag' and 'saga' fast convergence is only guaranteed on features with approximately the same scale. You can preprocess the data with a scaler from :mod:`sklearn.preprocessing`. .. seealso:: Refer to the User Guide for more information regarding :class:`LogisticRegression` and more specifically the Table summarizing solver/penalty supports. .. versionadded:: 0.17 Stochastic Average Gradient descent solver. .. versionadded:: 0.19 SAGA solver. .. versionchanged:: 0.22 The default solver changed from 'liblinear' to 'lbfgs' in 0.22. .. versionadded:: 1.2 newton-cholesky solver. **Tolerance** (tol) Tolerance for stopping criteria. **Warm start** (warm_start) When set to True, reuse the solution of the previous call to fit as initialization, otherwise, just erase the previous solution. Useless for liblinear solver. See warm_start. .. versionadded:: 0.17 *warm_start* to support *lbfgs*, *newton-cg*, *sag*, *saga* solvers. Implementation ============== .. automodule:: node_regression :noindex: .. class:: LogisticRegression :noindex: